<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  <meta http-equiv="Content-Style-Type" content="text/css" />
  <meta name="generator" content="pandoc" />
  <meta name="version" content="S5 1.1" />
  <meta name="author" content="" />
  <title>Data Analysis</title>
  <style type="text/css">
      code{white-space: pre-wrap;}
      span.smallcaps{font-variant: small-caps;}
      span.underline{text-decoration: underline;}
      div.column{display: inline-block; vertical-align: top; width: 50%;}
  </style>
  <!-- configuration parameters -->
  <meta name="defaultView" content="slideshow" />
  <meta name="controlVis" content="hidden" />
  <!-- style sheet links -->
  <link rel="stylesheet" href="www/slides.css" type="text/css" media="projection" id="slideProj" />
  <link rel="stylesheet" href="www/outline.css" type="text/css" media="screen" id="outlineStyle" />
  <link rel="stylesheet" href="www/print.css" type="text/css" media="print" id="slidePrint" />
  <link rel="stylesheet" href="www/opera.css" type="text/css" media="projection" id="operaFix" />
  <!-- S5 JS -->
  <script src="www/slides.js" type="text/javascript"></script>
</head>
<body>
<div class="layout">
<div id="controls"></div>
<div id="currentSlide"></div>
<div id="header"></div>
<div id="footer">
  <h1></h1>
  <h2>Data Analysis</h2>
</div>
</div>
<div class="presentation">
<div class="title-slide slide">
  <h1 class="title">Data Analysis</h1>
</div>
<div id="data-analysis" class="slide section level1">
<h1>Data analysis</h1>
<ul>
<li>Input
<ul>
<li>One or multiple constructed datasets</li>
</ul></li>
<li>Steps
<ul>
<li><strong>Exploratory analysis:</strong> explore, describe and look for patterns in the data</li>
<li><strong>Final data analysis:</strong> create and export polished tables and graphs</li>
</ul></li>
<li>Output
<ul>
<li>Excel tables</li>
<li>PNG or JPG images</li>
<li>Documentation</li>
</ul></li>
</ul>

</div>
<div id="exploratory-analysis" class="slide section level1">
<h1>Exploratory analysis</h1>
<ul>
<li>During exploratory analysis, we will explore the data and look for patterns</li>
<li>At this stage, we are looking to understand the data rather than to present findings</li>
<li>We will create code, but not necessarily export outputs or spend time making sure graphs and tables are well-presented</li>
</ul>
</div>
<div id="exploratory-analysis-setting-the-stage" class="slide section level1">
<h1>Exploratory analysis: setting the stage</h1>
<ol style="list-style-type: decimal">
<li>Open a new do-file in the do-file editor</li>
<li>Load the dataset that you want to explore</li>
</ol>

<p><code>use "../DataWork/Data/Final/final_process.dta", clear</code></p>
<p>Note that:</p>
<ul>
<li>We have created multiple constructed datasets, with different levels of observation</li>
<li>We will explore each of them separately</li>
<li>If you want to explore variables that are not in the dataset yet, go back to construction and create them</li>
</ul>
</div>
<div id="exploratory-analysis-useful-commands" class="slide section level1">
<h1>Exploratory analysis: useful commands</h1>
<ul>
<li><code>codebook</code>: to describe the dataset and individual variables</li>
<li><code>summarize</code>: to see descriptive statistics for continuous variables</li>
<li><code>tabulate</code>: to create frequency tables</li>
<li><code>tabstat</code>: to create summary statistics for multiple variables</li>
<li><code>histogram</code>: to create a histograms</li>
<li><code>kdensity</code>: to create a density graph for continuous variables</li>
<li><code>scatter</code>: to quickly visualize the relationship between two variables</li>
</ul>
</div>
<div id="exploratory-analysis-codebook" class="slide section level1">
<h1>Exploratory analysis: <code>codebook</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>codebook [varlist] [if] [in] [, options]</code></pre>
<ul>
<li>Explore the entire dataset:</li>
</ul>
<pre><code>codebook</code></pre>
<ul>
<li>Explore selected variables:</li>
</ul>
<pre><code>codebook varlist</code></pre>
</div>
<div id="exploratory-analysis-codebook-1" class="slide section level1">
<h1>Exploratory analysis: <code>codebook</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>codebook [varlist] [if] [in] [, options]</code></pre>
<p>Explore the specific variables:</p>
<pre class='stata'>. codebook process_value process_type

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
process_value                                                                                           Process Value
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

                  type:  numeric (float)

                 range:  [294.8,2.043e+09]            units:  .01
         unique values:  7,272                    missing .:  0/25,000

                  mean:   4.5e+06
              std. dev:   3.6e+07

           percentiles:        10%       25%       50%       75%       90%
                            250000    400000    848000   2.1e+06   6.0e+06

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
process_type                                                                                          Processing Type
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

                  type:  numeric (long)
                 label:  process_type

                 range:  [1,3]                        units:  1
         unique values:  3                        missing .:  8/25,000

            tabulation:  Freq.   Numeric  Label
                         1,691         1  Closed
                        21,043         2  Open
                         2,258         3  Restricted
                             8         .  
</pre>
</div>
<div id="exploratory-analysis-summarize" class="slide section level1">
<h1>Exploratory analysis: <code>summarize</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>summarize [varlist] [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> Use <code>summarize</code> to explore the variables <code>process_value</code> and <code>process_type</code></p>
</div>
<div id="exploratory-analysis-summarize-1" class="slide section level1">
<h1>Exploratory analysis: <code>summarize</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>summarize [varlist] [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> Use <code>summarize</code> to explore the variables <code>process_value</code> and <code>process_type</code></p>
<pre class='stata'>. summarize process_value process_type

    Variable │        Obs        Mean    Std. Dev.       Min        Max
─────────────┼─────────────────────────────────────────────────────────
process_va~e │     25,000     4478843    3.60e+07      294.8   2.04e+09
process_type │     24,992    2.022687    .3968654          1          3
</pre>
</div>
<div id="exploratory-analysis-summarize-2" class="slide section level1">
<h1>Exploratory analysis: <code>summarize</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>summarize [varlist] [if] [in] [weight] [, options]</code></pre>
<ul>
<li>Using the option <code>detail</code> will include more information about the distribution</li>
</ul>
<pre class='stata'>. summarize process_value, detail

                        Process Value
─────────────────────────────────────────────────────────────
      Percentiles      Smallest
 1%     63285.75          294.8
 5%       200000         464.79
10%       250000         990.23       Obs              25,000
25%       400000        1414.61       Sum of Wgt.      25,000

50%       848000                      Mean            4478843
                        Largest       Std. Dev.      3.60e+07
75%      2060000       1.78e+09
90%      6000000       1.90e+09       Variance       1.29e+15
95%     1.26e+07       2.04e+09       Skewness       40.09512
99%     6.00e+07       2.04e+09       Kurtosis       1987.837
</pre>
</div>
<div id="exploratory-analysis-tabulate-one-way" class="slide section level1">
<h1>Exploratory analysis: tabulate one way</h1>
<p><span style="font-size:10%"></span></p>
<pre><code>tabulate varname [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> use <code>tabulate</code> to explore the variable <code>procurement_type</code></p>
</div>
<div id="exploratory-analysis-tabulate-one-way-1" class="slide section level1">
<h1>Exploratory analysis: tabulate one way</h1>
<p><span style="font-size:10%"></span></p>
<pre><code>tabulate varname [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> use <code>tabulate</code> to explore the variable <code>procurement_type</code></p>
<pre class='stata'>. tabulate procurement_type

Procurement │
       Type │      Freq.     Percent        Cum.
────────────┼───────────────────────────────────
      Goods │     11,150       44.63       44.63
   Services │      8,119       32.50       77.12
      Works │      5,716       22.88      100.00
────────────┼───────────────────────────────────
      Total │     24,985      100.00
</pre>
</div>
<div id="exploratory-analysis-tabulate-two-way" class="slide section level1">
<h1>Exploratory analysis: tabulate two way</h1>
<p><span style="font-size:10%"></span></p>
<pre><code>tabulate varname1 varname2 [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> use <code>tabulate</code> to explore the relationship between variables <code>procurement_type</code> and <code>process_type</code></p>
</div>
<div id="exploratory-analysis-tabulate-two-way-1" class="slide section level1">
<h1>Exploratory analysis: tabulate two way</h1>
<p><span style="font-size:10%"></span></p>
<pre><code>tabulate varname1 varname2 [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> use <code>tabulate</code> to explore the relationship between variables <code>procurement_type</code> and <code>process_type</code></p>
<pre class='stata'>. tabulate procurement_type process_type

Procuremen │         Processing Type
    t Type │    Closed       Open  Restricte │     Total
───────────┼─────────────────────────────────┼──────────
     Goods │       615      9,593        940 │    11,148 
  Services │       781      6,315      1,020 │     8,116 
     Works │       295      5,120        298 │     5,713 
───────────┼─────────────────────────────────┼──────────
     Total │     1,691     21,028      2,258 │    24,977 
</pre>
</div>
<div id="exploratory-analysis-tabstat" class="slide section level1">
<h1>Exploratory analysis: <code>tabstat</code></h1>
<pre><code>tabstat varlist [if] [in] [weight] [, options]</code></pre>
<ul>
<li><code>tabstat</code> is a useful command that can provide summary statistics for a series of numeric variables in one table.<br />
</li>
<li>It allows you to specify the list of statistics to be displayed.<br />
</li>
<li>Statistics can be calculated (conditioned on) another variable.</li>
</ul>
</div>
<div id="exploratory-analysis-tabstat-1" class="slide section level1">
<h1>Exploratory analysis: <code>tabstat</code></h1>
<p>Let’s expore the minimum, maximum, mean, median, and count for the following variables: <code>process_value</code>, <code>nr_participants</code></p>
<pre class='stata'>. tabstat process_value nr_participants, stat(mean p50 n)

   stats │  proce~ue  nr_par~s
─────────┼────────────────────
    mean │   4478843  2.450993
     p50 │    848000         2
       N │     25000     22548
─────────┴────────────────────
</pre>
<p>Are there any other statistics we can explore through this command? What are they?</p>
</div>
<div id="exploratory-analysis-tabstat-var-byvar" class="slide section level1">
<h1>Exploratory analysis: <code>tabstat [var], by(var)</code></h1>
<p>What if we wanted to see these statistics for process_value and nr_participants over some other categorical variable?</p>
<pre class='stata'>. tabstat process_value nr_participants, stat(mean p50 n) by(process_type)

Summary statistics: mean, p50, N
  by categories of: process_type (Processing Type)

process_type │  proce~ue  nr_par~s
─────────────┼────────────────────
      Closed │  577892.7  1.521619
             │    150000         1
             │      1691      1087
─────────────┼────────────────────
        Open │   4721041  2.581185
             │  963868.6         2
             │     21043     19314
─────────────┼────────────────────
  Restricted │   5010361  1.750233
             │    656750         1
             │      2258      2146
─────────────┼────────────────────
       Total │   4466848  2.451013
             │    846000         2
             │     24992     22547
─────────────┴────────────────────
</pre>
</div>
<div id="exploratory-analysis-list-varlist" class="slide section level1">
<h1>Exploratory analysis: <code>list varlist</code></h1>
<ul>
<li><code>list</code> is a useful command that can display values of variables.</li>
</ul>
<p>Let’s list <code>process_values</code> for the first five observations in the dataset</p>
<pre class='stata'>. list process_id process_value in 1/5

     ┌────────────────────────────┐
     │   process_id   process_v~e │
     ├────────────────────────────┤
  1. │ DIR-2017-002   800,000,000 │
  2. │     EVV 1/17    21,607,448 │
  3. │ DIR-2020-002    63,347,800 │
  4. │       22/17.    17,701,036 │
  5. │     EP-26/20       550,000 │
     └────────────────────────────┘
</pre>
</div>
<div id="exploratory-analysis-list-varlist-1" class="slide section level1">
<h1>Exploratory analysis: <code>list varlist</code></h1>
<p>We can also list the top 5 entities with highest total process values by using <code>collapse</code></p>
<pre class='stata'>. collapse (sum) process_value, by(entity)

. gsort -process_value

. list entity process_value in 1/5

     ┌───────────────────────────────────────────────────────────────────┐
     │                                            entity   process_value │
     ├───────────────────────────────────────────────────────────────────┤
  1. │                                       GRAD ZAGREB   9,279,500,348 │
  2. │                      HRVATSKE CESTE D.O.O. ZAGREB   8,580,754,751 │
  3. │                          HŽ INFRASTRUKTURA D.O.O.   5,670,633,077 │
  4. │                         HRVATSKE AUTOCESTE D.O.O.   3,603,078,854 │
  5. │ HRVATSKE VODE, PRAVNA OSOBA ZA UPRAVLJANJE VODAMA   3,050,996,577 │
     └───────────────────────────────────────────────────────────────────┘
</pre>
</div>
<div id="exploratory-analysis-list-varlist-exercise" class="slide section level1">
<h1>Exploratory analysis: <code>list varlist</code> Exercise</h1>
<p><strong>Exercise</strong> Can you list the top 5 counties with the highest total process values? The variable for counties is <code>entity_counties</code></p>
<p>First open the <code>final_process.dta</code> again by running the following code</p>
<pre class='stata'>. use "../DataWork/Data/Final/final_process.dta", clear
</pre>
<p><code>use "DataWork/Data/Final/final_process.dta", clear</code></p>
</div>
<div id="exploratory-analysis-list-varlist-exercise-1" class="slide section level1">
<h1>Exploratory analysis: <code>list varlist</code> Exercise</h1>
<p><strong>Exercise</strong> Can you list the top 5 counties with the highest total process values? The variable for counties is <code>entity_counties</code></p>

<pre><code>use &quot;DataWork/Data/Final/final_process.dta&quot;, clear
collapse (sum) process_value, by(entity_county)
gsort -process_value
list entity_county process_value in 1/5</code></pre>
<p>How will be list the counties with the lowest total process values?</p>
</div>
<div id="exploratory-analysis-histogram" class="slide section level1">
<h1>Exploratory analysis: <code>histogram</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>histogram varname [if] [in] [weight] [, [continuous_opts | discrete_opts] options]</code></pre>
<p><strong>Exercise:</strong> use <code>histogram</code> to explore the distibrution of variable <code>month_init</code></p>

<p><code>use "DataWork/Data/Final/final_process.dta", clear</code></p>
<p><img src="img/hist_month_init.png" style="width:40.0%" /></p>
</div>
<div id="exploratory-analysis-histogram-1" class="slide section level1">
<h1>Exploratory analysis: <code>histogram</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>histogram varname [if] [in] [weight] [, [continuous_opts | discrete_opts] options]</code></pre>
<p><strong>Exercise:</strong> use <code>histogram</code> to explore the distibrution of variable <code>month_init</code></p>
<pre><code>histogram month_init</code></pre>
<p><img src="img/hist_month_init.png" style="width:40.0%" /></p>
</div>
<div id="exploratory-analysis-histogram-2" class="slide section level1">
<h1>Exploratory analysis: <code>histogram</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>histogram varname [if] [in] [weight] [, [continuous_opts | discrete_opts] options]</code></pre>
<pre><code>histogram month_init, discrete</code></pre>
<p><img src="img/hist_month_init_discrete.png" style="width:40.0%" /></p>
</div>
<div id="exploratory-analysis-kdensity" class="slide section level1">
<h1>Exploratory analysis: <code>kdensity</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>kdensity varname [if] [in] [weight] [, options]</code></pre>
<pre><code>kdensity bid_submission_period</code></pre>
<p><img src="img/density_bid_period.png" style="width:40.0%" /></p>
</div>
<div id="exploratory-analysis-kdensity-1" class="slide section level1">
<h1>Exploratory analysis: <code>kdensity</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>kdensity varname [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> use <code>kdensity</code> to explore the distribution of variable <code>bid_submission_period</code> only for processes where the period is shorter than 100 days</p>
<p><img src="img/density_bid_period_restricted.png" style="width:30.0%" /></p>
</div>
<div id="exploratory-analysis-kdensity-2" class="slide section level1">
<h1>Exploratory analysis: <code>kdensity</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>kdensity varname [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> use <code>kdensity</code> to explore the distribution of variable <code>bid_submission_period</code> only for processes where the period is shorter than 100 days</p>
<pre><code>kdensity bid_submission_period if bid_submission_period &lt; 100</code></pre>
<p><img src="img/density_bid_period_restricted.png" style="width:30.0%" /></p>
</div>
<div id="exploratory-analysis-scatter" class="slide section level1">
<h1>Exploratory analysis: <code>scatter</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>scatter varlist [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> use <code>scatter</code> to explore the relationship between variables <code>nr_lots</code> and <code>nr_participants</code></p>
<p><img src="img/scatter.png" style="width:30.0%" /></p>
</div>
<div id="exploratory-analysis-scatter-1" class="slide section level1">
<h1>Exploratory analysis: <code>scatter</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>scatter varlist [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> use <code>scatter</code> to explore the relationship between variables <code>nr_lots</code> and <code>nr_participants</code></p>
<pre><code>scatter nr_lots nr_participants</code></pre>
<p><img src="img/scatter.png" style="width:30.0%" /></p>
</div>
<div id="exploratory-analysis-graph-bar" class="slide section level1">
<h1>Exploratory analysis: <code>graph bar</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>graph bar  yvars [if] [in] [weight] [, options]
graph hbar yvars [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> use <code>graph bar</code> to explore the variable <code>process_value</code></p>
<p><img src="img/process_value.png" style="width:30.0%" /></p>
</div>
<div id="exploratory-analysis-graph-bar-1" class="slide section level1">
<h1>Exploratory analysis: <code>graph bar</code></h1>
<p><strong>Exercise:</strong> use <code>graph bar</code> to explore the variable <code>process_value</code></p>
<pre><code>graph bar process_value</code></pre>
<p><img src="img/process_value.png" style="width:30.0%" /></p>
<p><span style="border-left: solid 5px lightgray;padding-left: 1em;display: block;margin-block-start: 1em;margin-block-end: 1em;margin-inline-start: 40px;margin-inline-end: 40px;font-size:80%">Note that by default a bar graph displays the average value of a continuous variable.</span></p>
</div>
<div id="exploratory-analysis-graph-bar-2" class="slide section level1">
<h1>Exploratory analysis: <code>graph bar</code></h1>
<p>Use the option <code>over()</code> to break down a bar graph into groups</p>
<pre><code>graph bar process_value, over(procurement_type){width=40%}</code></pre>
<p><img src="img/process_value_proc_type.png" style="width:40.0%" /></p>
</div>
<div id="exploratory-analysis-graph-bar-3" class="slide section level1">
<h1>Exploratory analysis: <code>graph bar</code></h1>
<p>Write <code>(stat)</code>, where <code>stat</code> represents the statistic to be calculated, before a variable’s name to show a statistic other than the mean</p>
<pre><code>graph bar (sum) process_value, over(procurement_type)</code></pre>
<p><img src="img/process_value_proc_type_total.png" style="width:40.0%" /></p>
</div>
<div id="exploratory-analysis-graph-bar-4" class="slide section level1">
<h1>Exploratory analysis: <code>graph bar</code></h1>
<ul>
<li>Possible values of <code>stat</code> are:
<ul>
<li><code>sum</code></li>
<li><code>count</code></li>
<li><code>mean</code></li>
<li><code>median</code></li>
<li><code>percent</code></li>
</ul></li>
</ul>
</div>
<div id="exploratory-analysis-graph-bar-5" class="slide section level1">
<h1>Exploratory analysis: <code>graph bar</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>graph bar  yvars [if] [in] [weight] [, options]
graph hbar yvars [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> Create a horizontal bar graph that shows the number of processes with each procurement type</p>
<p><img src="img/process_type_count.png" style="width:30.0%" /></p>
</div>
<div id="exploratory-analysis-graph-bar-6" class="slide section level1">
<h1>Exploratory analysis: <code>graph bar</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>graph bar  yvars [if] [in] [weight] [, options]
graph hbar yvars [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> Create a horizontal bar graph that shows the number of processes with each procurement type</p>
<pre><code>graph hbar (count) process_id, over(procurement_type)</code></pre>
<p><img src="img/process_type_count.png" style="width:35.0%" /></p>
</div>
<div id="exploratory-analysis-graph-bar-7" class="slide section level1">
<h1>Exploratory analysis: <code>graph bar</code></h1>
<pre><code>graph bar  yvars [if] [in] [weight] [, options]
graph hbar yvars [if] [in] [weight] [, options]</code></pre>
<p><strong>Exercise:</strong> Create a horizontal bar graph that shows the top 5 procuring entities with the highest total process value</p>
<p><span style="border-left: solid 5px lightgray;padding-left: 1em;display: block;margin-block-start: 1em;margin-block-end: 1em;margin-inline-start: 40px;margin-inline-end: 40px;font-size:80%"><strong>Tip:</strong> Refer to our exercise on listing the top 5 procuring entities</span></p>
<p><img src="img/topcounties.png" style="width:35.0%" /></p>
</div>
<div id="exploratory-analysis-graph-bar-8" class="slide section level1">
<h1>Exploratory analysis: <code>graph bar</code></h1>
<p><strong>Exercise:</strong> Create a horizontal bar graph that shows the top 5 counties with the highest total process value</p>
<pre><code>use &quot;DataWork/Data/Final/final_process.dta&quot;, clear

collapse (sum) process_value, by(entity_county) // collapses your data at county level

gsort -process_value // sorts total process_value in descending order

gen rank = _n // generates a new variable that ranks the observations after sorting

graph hbar (sum) process_value if rank &lt;= 5, over(entity_county) // creates a graph of top counties </code></pre>

</div>
<div id="final-analysis" class="slide section level1">
<h1>Final analysis</h1>
<ul>
<li>Once we have explored the data and understood what it is showing us, we will decide how to present our results</li>
<li>Formatting outputs to make sure they include all relevant information can be very time-consuming</li>
<li>We will only spend time formatting the outputs that we want to share with others</li>
</ul>
</div>
<div id="final-analysis-setting-the-stage" class="slide section level1">
<h1>Final analysis: setting the stage</h1>
<p><strong>Exercise:</strong></p>
<p><strong>1.</strong> Save the code that you wrote for exploratory analysis on the <code>Code/Analysis</code> folder. Call it <code>explore-process-data.do</code>.</p>
<p><strong>2.</strong> Open a new script</p>
<p><strong>3.</strong> Load the data that you want to use</p>
<p><strong>4.</strong> Copy the code that created the graph you want to polish</p>
</div>
<div id="final-analysis-graph-bar" class="slide section level1">
<h1>Final analysis: <code>graph bar</code></h1>
<p><span style="font-size:10%"></span></p>

<pre><code>Untitled do-file -----------------------------------

use &quot;../DataWork/Data/Final/final_process.dta&quot;

graph bar (count) process_id, over(procurement_type) </code></pre>
</div>
<div id="final-analysis-graph-bar-1" class="slide section level1">
<h1>Final analysis: <code>graph bar</code></h1>
<p>To change the axis title, use <code>xtitle()</code> or <code>ytitle()</code></p>
<pre><code>graph bar (count) process_id, ///
    over(procurement_type) ///
    ytitle(&quot;Number of processes&quot;) &lt;----</code></pre>
<p><img src="img/title.png" style="width:35.0%" /></p>
</div>
<div id="final-analysis-graph-bar-2" class="slide section level1">
<h1>Final analysis: <code>graph bar</code></h1>
<p>To add labels to the bars, use the option <code>blabel()</code></p>
<pre><code>graph bar (count) process_id, ///
    over(procurement_type) ///
    ytitle(&quot;Number of processes&quot;) ///
    blabel(total) &lt;----</code></pre>
<p><img src="img/blabel.png" style="width:35.0%" /></p>
</div>
<div id="final-analysis-graph-bar-3" class="slide section level1">
<h1>Final analysis: <code>graph bar</code></h1>
<p>To change the background color, use the option <code>graphregion(color())</code></p>
<pre><code>graph bar (count) process_id, ///
    over(procurement_type) ///
    ytitle(&quot;Number of processes&quot;) ///
    blabel(total)  ///
    graphregion(color(white)) &lt;----</code></pre>
<p><img src="img/grregion.png" style="width:35.0%" /></p>
</div>
<div id="final-analysis-set-scheme" class="slide section level1">
<h1>Final analysis: <code>set scheme</code></h1>
<p>An easier way to format graphs is to use pre-defined color schemes. Use the <code>set scheme</code> command to select a color scheme to use</p>
<pre><code>ssc install blindschemes, replace
set scheme plotplain

graph bar (count) process_id, ///
    over(procurement_type) ///
    ytitle(&quot;Number of processes&quot;) ///
    blabel(total) </code></pre>
<p><img src="img/plotplain.png" style="width:30.0%" /></p>
</div>
<div id="final-analysis-set-scheme-1" class="slide section level1">
<h1>Final analysis: <code>set scheme</code></h1>
<pre><code>set scheme white_tableau // See https://github.com/asjadnaqvi/Stata-schemes for more schemes

graph bar (count) process_id, ///
    over(procurement_type) ///
    ytitle(&quot;Number of processes&quot;) ///
    blabel(total) </code></pre>
<p><img src="img/white_tableau.png" style="width:30.0%" /></p>
</div>
<div id="final-analysis-graph-bar-4" class="slide section level1">
<h1>Final analysis: <code>graph bar</code></h1>
<p><strong>Exercise:</strong> recreate the graph shown below.</p>
<p><span style="border-left: solid 5px lightgray;padding-left: 1em;display: block;margin-block-start: 1em;margin-block-end: 1em;margin-inline-start: 40px;margin-inline-end: 40px;font-size:80%"><strong>Tip:</strong> the color scheme is called <code>white_w3d</code></span></p>
<p><img src="img/total_value_proctype_bar.png" style="width:30.0%" /></p>
</div>
<div id="final-analysis-graph-bar-5" class="slide section level1">
<h1>Final analysis: <code>graph bar</code></h1>
<p><strong>Exercise:</strong> recreate the graph shown below</p>
<pre><code>set scheme white_w3d

gr bar (sum) process_value, ///
    over(procurement_type) ///
    ytitle(&quot;Awarded value (Lev)&quot;) ///
    blabel(total) ///
    title(&quot;Total awarded value by process type&quot;)</code></pre>
<p><img src="img/total_value_proctype_bar.png" style="width:30.0%" /></p>
</div>
<div id="final-analysis-exporting-outputs" class="slide section level1">
<h1>Final analysis: exporting outputs</h1>
<ul>
<li>Always save outputs in easily accessible format (such as PNG ang JPEG)</li>
<li>Use self-explanatory names on your graphs</li>
<li>Graph names and the names of the do-files that create them should be easy to match</li>
</ul>
</div>
<div id="final-analysis-exporting-graphs" class="slide section level1">
<h1>Final analysis: exporting graphs</h1>
<p><span style="font-size:10%"></span></p>
<pre><code>graph export newfilename.suffix [, options]</code></pre>
<p><strong>Exercise:</strong> Export the graph you just created to the <code>Output</code> folder. Call it <code>process-value-by-procurement-type.png</code>.</p>
</div>
<div id="final-analysis-exporting-graphs-1" class="slide section level1">
<h1>Final analysis: exporting graphs</h1>
<p><span style="font-size:10%"></span></p>
<pre><code>graph export newfilename.suffix [, options]</code></pre>
<p><strong>Exercise:</strong> Export the graph you just created to the <code>Output</code> folder. Call it <code>process-value-by-procurement-type.png</code>.</p>
<pre><code>graph export &quot;../DataWork/Output/process-value-by-procurement-type.png&quot;, replace</code></pre>
</div>
<div id="final-analysis-saving-do-files" class="slide section level1">
<h1>Final analysis: saving do-files</h1>
<p>Final analysis scripts should be short and simple:</p>
<p><strong>1.</strong> Load the data to be used</p>
<p><strong>2.</strong> Create the graph or table to be exported</p>
<p><strong>3.</strong> Export the graph or table</p>
<p><strong>Exercise:</strong> Clean and save the do-file that created your graph to the folder <code>Code/Analysis</code>. Call it <code>process-value-by-procurement-type.do</code></p>
</div>
<div id="final-analysis-saving-do-files-1" class="slide section level1">
<h1>Final analysis: saving do-files</h1>
<p><code>process-value-by-procurement-type.do</code></p>
<pre><code>set scheme white_w3d

use &quot;../DataWork/Data/Final/final_process.dta&quot;

gr bar (sum) process_value, ///
    over(procurement_type) ///
    ytitle(&quot;Awarded value (Lev)&quot;) ///
    blabel(total)  

graph export &quot;../DataWork/Output/process-value-by-procurement-type.png&quot;, replace</code></pre>
</div>
<div id="final-analysis-saving-do-files-2" class="slide section level1">
<h1>Final analysis: saving do-files</h1>
<p><code>processes-by-procurement-type.do</code></p>
<pre><code>set scheme white_w3d

use &quot;../DataWork/Data/Final/final_process.dta&quot;

gr bar (sum) process_id, ///
    over(procurement_type) ///
    ytitle(&quot;Number of processes&quot;) ///
    blabel(total)  

graph export &quot;../DataWork/Output/processes-by-procurement-type.png&quot;, replace</code></pre>
</div>
<div id="final-analysis-saving-do-files-3" class="slide section level1">
<h1>Final analysis: saving do-files</h1>
<p>Analysis scripts should be this short for a few reasons:</p>
<p><strong>1.</strong> In the future, we may only want to recreate one graph and not all of the graph we ever exported. In these cases, running a short script is much faster.</p>
<p><strong>2.</strong> We want to be able to quickly find the piece of code that created an image. It is harder to read a long script and find the exact part of the code that created it than it is to connect a graph name to a do-file name.</p>
<p>In the next session, we will see how to recreate many figures without opening and running each do-file individually.</p>
</div>
<div id="final-analysis-graph-pie" class="slide section level1">
<h1>Final analysis: <code>graph pie</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>Slices as totals or percentages within over() categories
    graph pie varname [if] [in] [weight], over(varname) [options]

Slices as frequencies within over() categories
    graph pie [if] [in] [weight], over(varname) [options]</code></pre>
</div>
<div id="final-analysis-graph-pie-1" class="slide section level1">
<h1>Final analysis: <code>graph pie</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>graph pie, over(procurement_type)</code></pre>
<p><img src="img/pie1.png" style="width:40.0%" /></p>
</div>
<div id="final-analysis-graph-pie-2" class="slide section level1">
<h1>Final analysis: <code>graph pie</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>graph pie process_value, over(procurement_type)</code></pre>
<p><img src="img/pie2.png" style="width:40.0%" /></p>
</div>
<div id="final-analysis-graph-pie-3" class="slide section level1">
<h1>Final analysis: <code>graph pie</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>gr pie process_value, over(procurement_type) plabel(_all percent)</code></pre>
<p><img src="img/pie3.png" style="width:40.0%" /></p>
</div>
<div id="final-analysis-graph-pie-4" class="slide section level1">
<h1>Final analysis: <code>graph pie</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>gr pie process_value, over(procurement_type) plabel(_all percent) legend(position(12) cols(3))</code></pre>
<p><img src="img/pie4.png" style="width:40.0%" /></p>
</div>
<div id="final-analysis-graph-pie-5" class="slide section level1">
<h1>Final analysis: <code>graph pie</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>gr pie process_value, over(procurement_type) plabel(_all percent) legend(position(4) cols(1))</code></pre>
<p><img src="img/pie5.png" style="width:40.0%" /></p>
</div>
<div id="final-analysis-graph-pie-6" class="slide section level1">
<h1>Final analysis: <code>graph pie</code></h1>
<p><span style="font-size:10%"></span></p>
<pre><code>gr pie process_value, over(procurement_type) plabel(_all percent, gap(10)) legend(on position(4) cols(1)) pie(2, explode)</code></pre>
<p><img src="img/pie6.png" style="width:40.0%" /></p>
</div>
<div id="two-way-graphs" class="slide section level1">
<h1>Two way graphs</h1>
<ul>
<li>All the graphs we’ve seen so far illustrate a single continuous variable, either by itself or over a categorical variable</li>
<li>To show the relationship between two continuous variables, however, another type of Stata graph is used: the two-way graph</li>
</ul>
<p>If we wanted to see how the total process value changes over time, for example, we could simple treat time as a categorical variable and create the following graph:</p>
<pre><code>gr bar (sum) process_value, over(year_init) </code></pre>
<p><img src="img/value_year.png" style="width:20.0%" /></p>
</div>
<div id="two-way-graphs-1" class="slide section level1">
<h1>Two way graphs</h1>
<ul>
<li>However, time trends are usually displayed as line plots, which are in Stata created as a two-way graph</li>
<li>In this two-way graph, we have two continuous variables: time and process value</li>
<li>When we create two-way graphs, the most important thing to have in mind is that the data used for the graph needs to be tidy</li>
<li>That is, each row in the dataset should represent one data point to be displayed in the graph – we cannot group observations by calculating the mean or percent only in the graph, as we did for one way graphs</li>
<li>In our example, in order to create a line plot of total process value by year, we need to start from a dataset where each row represents one year</li>
</ul>
</div>
<div id="two-way-graphs-2" class="slide section level1">
<h1>Two-way graphs</h1>
<p>The way to create this type-year level dataset was discussed when we talked about construction:</p>
<pre class='stata'>. collapse (sum) process_value, by(year_init procurement_type)
</pre>
<p>This will create the following dataset:</p>
<pre class='stata'>. codebook, compact

Variable      Obs Unique      Mean      Min       Max  Label
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
year_init      28      8  2017.643     2014      2021  Year Procedure Was Initiated
procuremen~e   27      3         2        1         3  Procurement Type
process_va~e   32     32  3.50e+09  1556786  1.17e+10  (sum) process_value
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
</pre>
</div>
<div id="two-way-graphs-3" class="slide section level1">
<h1>Two-way graphs</h1>
<p>Once the data is at the appropriate level of observation, we can create a two-way plot:</p>
<pre><code>[graph] twoway plottype yvar xvar [if] [in] [, twoway_options]</code></pre>
<pre><code>graph twoway line process_value year_init if procurement_type == 1</code></pre>
<p><img src="img/tw1.png" style="width:40.0%" /></p>
</div>
<div id="two-way-graphs-4" class="slide section level1">
<h1>Two-way graphs</h1>
<p>One nice freature of two-way graphs is that we can add layers to a plot:</p>
<pre><code>graph twoway ///
    (line process_value year_init if procurement_type == 1) /// First layer
    (line process_value year_init if procurement_type == 2) /// Second layer
    (line process_value year_init if procurement_type == 3) // Third layer</code></pre>
<p><img src="img/tw2.png" style="width:40.0%" /></p>
</div>
<div id="two-way-graphs-5" class="slide section level1">
<h1>Two-way graphs</h1>
<ul>
<li>When a graph has multiple layers, there are two different types of options we can add: layer-specific options and general graph options</li>
<li>General graph options are separated by a comma and added outside of the parentheses that define a layer</li>
</ul>
</div>
<div id="two-way-graphs-6" class="slide section level1">
<h1>Two-way graphs</h1>
<pre><code>graph twoway ///
    (line process_value year_init if procurement_type == 1, /// First layer
        lpattern(dash))                                     /// Option for first layer
    (line process_value year_init if procurement_type == 2) /// Second layer
    (line process_value year_init if procurement_type == 3), /// Third layer
    legend(order(1 &quot;Goods&quot; 2 &quot;Services&quot; 3 &quot;Works&quot;)) // General graph option</code></pre>
<p><img src="img/tw4.png" style="width:40.0%" /></p>
</div>
<div id="two-way-graphs-7" class="slide section level1">
<h1>Two-way graphs</h1>
<p><strong>Exercise:</strong> add a title to the graph that was created previously</p>
<p><img src="img/tw5.png" style="width:40.0%" /></p>
</div>
<div id="two-way-graphs-8" class="slide section level1">
<h1>Two-way graphs</h1>
<p><strong>Exercise:</strong> add a title to the graph that was created previously</p>
<pre><code>graph twoway ///
    (line process_value year_init if procurement_type == 1) /// First layer
    (line process_value year_init if procurement_type == 2) /// Second layer
    (line process_value year_init if procurement_type == 3), /// Third layer
    legend(order(1 &quot;Goods&quot; 2 &quot;Services&quot; 3 &quot;Works&quot;)) /// General graph option
    title(Total process value by over procurement type (HRK))    &lt;---- Title is a general graph option</code></pre>
<p><img src="img/tw5.png" style="width:40.0%" /></p>
</div>
<div id="two-way-graphs-9" class="slide section level1">
<h1>Two-way graphs</h1>
<ul>
<li>There is a large number of graph types that can be added as twoway graphs, including
<ul>
<li>Bar plots</li>
<li>Area plots</li>
<li>Line plots</li>
<li>Dropline plots</li>
</ul></li>
<li>To see a complete list of two-way types and their layer-specific options, type <code>help twoway</code></li>
</ul>
</div>
<div id="final-analysis-exporting-tables" class="slide section level1">
<h1>Final analysis: exporting tables</h1>
<ul>
<li>The <code>collapse</code> command that we just used to make sure our dataset was represeting year-level observations is also useful to create descriptive tables</li>
<li>The easiest way to export a descriptive table to Excel is to collapse the dataset, create the columns we want to present in our table, and export the entire dataset to Excel</li>
</ul>
<p><strong>Steps:</strong></p>
<p><strong>1.</strong> Load the constructed process dataset</p>
<p><strong>2.</strong> Collapse the data to display the desired statistics for each bid procedure</p>
<p><strong>3.</strong> Label the columns using label variable</p>
<p><strong>4.</strong> Export the resulting dataset to Excel</p>
</div>
<div id="exporting-tables" class="slide section level1">
<h1>Exporting tables</h1>
<p><span style="font-size:10%"></span></p>
<pre><code>use &quot;../DataWork/Data/Final/final_process.dta&quot;, clear

collapse (sum) value = process_value (percent) percent = process_value (count) volume = bid_id, by(bid_procedure)

label variable value     &quot;Awarded value (Lev)&quot;
label variable percent   &quot;Percent of awarded value&quot;
label variable volume    &quot;Volume of processes&quot;

export excel using &quot;../DataWork/Output/Tables.xls&quot;, sheet(&quot;Bid procedure&quot;) sheetreplace firstrow(varlabels)</code></pre>
</div>
<div id="final-analysis-exporting-tables-1" class="slide section level1">
<h1>Final analysis: exporting tables</h1>
<p><strong>Exercise:</strong> recreate the table shown below</p>
<p><img src="img/table2.png" style="width:40.0%" /></p>
<p><strong>1.</strong> Open a new do-file</p>
<p><strong>2.</strong> Load the constructed process dataset</p>
<p><strong>3.</strong> Collapse the data to display the desired statistics for each bid procedure</p>
<p><strong>4.</strong> Label the columns using label variable</p>
<p><strong>5.</strong> Export the resulting dataset to Excel</p>
</div>
<div id="final-analysis-exporting-tables-2" class="slide section level1">
<h1>Final analysis: exporting tables</h1>
<p><strong>Exercise:</strong> recreate the table shown below</p>
<p><img src="img/table2.png" style="width:40.0%" /></p>
<pre><code>use &quot;../DataWork/Data/Final/final_process.dta&quot;, clear
    
collapse (sum)        value = process_value ///
         (percent) percent = process_value ///
         (count)   volume = bid_id, ///
         by(bid_procedure)

label variable value     &quot;Awarded value (Lev)&quot;
label variable percent   &quot;Percent of awarded value&quot;
label variable volume    &quot;Volume of processes&quot;

export excel using &quot;../DataWork/Output/Tables.xls&quot;, sheet(&quot;Bid procedure&quot;) sheetreplace firstrow(varlabels)</code></pre>
</div>
</div>
</body>
</html>
